Recently, Deep Convolutional Neural Networks (DCNNs) have made unprecedentedprogress, achieving the accuracy close to, or even better than human-levelperception in various tasks. There is a timely need to map the latest softwareDCNNs to application-specific hardware, in order to achieve orders of magnitudeimprovement in performance, energy efficiency and compactness. StochasticComputing (SC), as a low-cost alternative to the conventional binary computingparadigm, has the potential to enable massively parallel and highly scalablehardware implementation of DCNNs. One major challenge in SC based DCNNs isdesigning accurate nonlinear activation functions, which have a significantimpact on the network-level accuracy but cannot be implemented accurately byexisting SC computing blocks. In this paper, we design and optimize SC basedneurons, and we propose highly accurate activation designs for the three mostfrequently used activation functions in software DCNNs, i.e, hyperbolictangent, logistic, and rectified linear units. Experimental results on LeNet-5using MNIST dataset demonstrate that compared with a binary ASIC hardware DCNN,the DCNN with the proposed SC neurons can achieve up to 61X, 151X, and 2Ximprovement in terms of area, power, and energy, respectively, at the cost ofsmall precision degradation.In addition, the SC approach achieves up to 21X and41X of the area, 41X and 72X of the power, and 198200X and 96443X of theenergy, compared with CPU and GPU approaches, respectively, while the error isincreased by less than 3.07%. ReLU activation is suggested for future SC basedDCNNs considering its superior performance under a small bit stream length.
展开▼
